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(54) Abstract Title 

A method of producing audibly discernible sounds 

(57) Means for producing an audibly discernable representation of a three dimensional structure such as a 
b.ologica structure comprises a preliminary analysis of the structure to determine three dimensional 
positional coordinate data, processing this data to filter out large scale and small scale variations into 
successive sets of data, combining this data into a single sequence and generating audible sounds therefrom 
relating to the positional coordinate data. The biological structure represented are proteins whereby the CA 

°J. the "!!*» acWs of th « P rot « in s are analysed by x-ray crystallography to produce the data sets 
tor generating the audible sounds. 
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1 

A METHOD OF PRODUCING AUDIBLY DISCERNIBLE SOUNDS 

The present invention relates generally to a method for producing audibly discernible 
sounds, and particularly to a method in which the sounds are representative of complex 
structures. 

In the representation of complex structures occupying three-dimensional space it has 
been conventional either to use stylised models of the structures, themselves occupying 
the three-dimensional space, or to represent the structures in two-dimensions with an 
appropriate representational convention by which the information concerning the third 
dimension is conveyed. 

Each of these representational techniques has inherent drawbacks. Three-dimensional 
models, although they can be produced accurately, are not readily transportable nor are 
they indefinitely extendable, so that, for example, a highly complex structure may be 
limited to a representation of only a portion under immediate examination. Although 
two-dimensional representations are more readily portable, and can also be converted 
into a form suitable for display on a screen such as a computer screen, the necessity for 
a drawing convention to represent the third dimension results in a degree of 
imprecision and uncertainty in the accuracy of the representation. The present 
invention proposes a complete departure from conventional representational techniques 
by proposing to provide an audibly discernible representation of spatial information. 

The present invention finds particular (but not exclusive) application in the 
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representation of naturally-occurring structures such as biological structures, especially 
proteins Structure, pattern and form are inherent in naturally-occurring biological 
material, and these features may be drawn upon to provide parameters for audibly 
discernible representations In the representation of spatial information in audible form 
the audible information may approximate to, or be considered as, music. Culture and 
musical tradition has resulted in the evolution of music consisting of references and 
structures often not consciously appreciated by the listener but nevertheless having an 
effect on the physiological perception of the music. Recent developments in digital 
computing have made available to musicians further means by which musical 
composition may be developed, and it is not unknown for a digital computer to be used 
as a compositional tool for developing new structures in music. This new type of 
music is sometimes called algorithmic, and such music, developed with computer 
process rules, may combine tonal and structural relationships which it would be 
difficult to calculate by traditional means. 

The audible representation of biological structures such as proteins may, therefore, 
result in sounds which may be musical in their form. Proteins are biological macro 
molecules which are the fundamental building blocks in all life forms. Proteins are 
composed of a plurality of sub units, amino acids, which are arranged according to the 
individual amino acid properties to form a three dimensional molecule. It is recognised 
that music is defined by the intervals between notes rather than the absolute pitches of 
the notes and, correspondingly, proteins may be considered as being denned more by 
their overall patterns than by their absolute or primary sequences. Proteins structures 
may be considered to comprise four mains elements or levels of complexity, namely a 
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primary structure which is the linear order in which the amino acid sub units occur. A 
secondary and a tertiary structure determined by the properties of these amino acids 
which give rise to structural features referred to by biologists as loops, turns, alpha 
helices and beta strands or beta sheets depending on the spatial arrangements of the sub 
units in the linear sequence. Quaternary structures are recognisable as the quaternary 
structure is the broadest structural level concerned with the arrangement of individual 
protein "blocks" which fit together to form the final protein molecule. 

At the primary level, alpha helices are uniform coil-like structures in which the 
functional groups or R-groups of the amino acid project outward from the helix axis. 
On the other hand secondary structures known as beta strands fold back and forth 
with individual amino acid R-groups projecting from the folded chain on alternate 
sides. Different beta strands may be aligned with one another, adjacent strands forming 
weak bonds which connect them into what are called beta-sheets. Also at the 
secondary/tertiary level regions of the molecule referred to as "turns" follow a different 
directional path and are commonly found connecting two regions of an alpha helix 
beta strands. These are structures spanning both the secondary and tertiary level 
that the folding patterns of the secondary structures are combined to produce the 
overall tertiary structure of a protein. Information concerning the relationship of these 
features is usually considered at a visual level and represented visually. The present 
invention provides an alternative means of representation allowing other senses to be 
involved. 



or 



in 



According to one aspect of the present invention, therefore, a method of producing 



an 
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audibly discernible representation of a three dimensional structure comprises the steps 
of: 

generating numerical information concerning the spatial position of identifiable 
components of the structure from analysis of the structure, 

converting the numerical information into a derived digital signal by forming 
moving averages of the numerical information, 

subjecting the derived digital signal to a conversion from the three-dimensional 
value to produce a signal having a scalar value representing at least one of the 
properties of a audibly discernible sound, and 

generating a sound having that or those properties. 

At a fundamental level it may be considered that each positional coordinate 
representing the presence of a feature or structural element of the structure may be 
allocated a unique discernible sound. It is of course important that the same sound is 
repeatably produced by the same feature, but it is not essential that only one feature 
produce that sound.applied to the representation of a naturally occurring structure, 
such as a protein the analysis may be performed by x-ray crystallography to determine 
the relative locations of the amino acids of which the protein is composed. In its 
application to other structures, however, different analysis techniques may be used on 
order to generate the raw positional data and, of course, the coordinate information 
may represent the positions of other significant elements in the structure. In an 
engineering structure, for example, the positional coordinates may represent the nodes 
in a load-bearing assembly and/or the stresses applied to elements of a structure during 
operation or when performing its function. For example, the stresses exerted on an 
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element of a building may be represented by pitch, variation of which represents 
increases in the stress so that dynamic variation of the stress pattern can be represented 
audibly which enables a plurality of different signals to be perceived simultaneously 
even in a structure in which, with a traditional visual representation, certain 
components may be masked or covered by others and therefore not visible. 

In the representation of structures such as proteins the resultant data of the x-ray 
crystallography analysis may comprise a set of positional coordinates representing the 
relative positions of at least some of the atoms of the amino acids of the protein. In 
particular the positional coordinate data may represent the three dimensional positions 
in space, with respect to a given frame of reference, of the CA carbon atoms of the 
amino acids of the protein. 

Although individual unique sounds may be generated these sounds may not necessarily 
be represented by individual musical notes, but could be other detectable sounds such 
as chords or discordant sounds. 

In the performance of the method of the invention the said moving averages comprise 
at least a first moving average of the value of the positional coordinate data scanned 
over successive groups of carbon atoms to provide an approximate representation of 
the secondary structure of the amino acids in the protein. 

Likewise, a second moving average of the values of the first moving average may be 
taken over a range of values thereof greater than the range used to form the first 
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moving average. This second moving average provides a further degree of 
"smoothing" to act effectively to filter out the smaller scale variations, without 
completely excluding larger scale variations. 

5 Further sequences of digital signals may be derived from the said first and second 
moving averages, for example by determining the differences between the first moving 
average and the positional coordinate data, whereby to provide a digital data signal in 
which larger scale variations are not present and smaller scale variations are amplified. 

10 The second further sequences of digital signals may be derived from the first and 
second moving averages by determining the differences between the first and second 
moving averages whereby, in effect, to act as a mid-frequency filter on the data. 

The method may include deriving a third further sequence of digital signals by 
1 5 normalising the second moving average values about a selected origin to represent the 
large scale features of the positional coordinate data stream. This, effectively, acts as 
a low frequency filter to produce signal dependent on the large scale features. 

The said first, second and third further sequences of digital signals may each be 
20 multiplied by a respective weighting factor and summed to produce a single sequence 
of averaged and processed three dimensional coordinate values which are then mapped 
onto a sequence of one-dimensional values. This may be achieved via a three- 
dimensional wave form generated by spectral synthesis, using a fourier synthesis 
technique, or by other techniques which provide a repeatable mapping of a three- 
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dimensional point onto a one-dimensional point which can be interpreted as a musical 
parameter. A simple example might be to sum together the three coordinate values to 
get a single one-dimensional value. 

The one-dimensional value may be pitch or volume and other means may be provided 
for varying the attack and decay of the sound in order to generate musical notes. 

The present invention also comprehends a method of producing an audibly discernible 
representation of a three-dimensional structure in which the identification of the details 
of secondary structures in the positional coordinate data is obtained by means of a 
filter. 

In another aspect of the invention a method of producing audibly recognisable sounds 
representing respective amino acids in a protein structure comprises the steps of: 
subjecting the protein structure to an x-ray crystallography analysis to produce a series 
of raw data signals representing the three-dimensional positional coordinates of the CA 
carbon atoms of the amino acids of which the protein is composed; filtering the said 
raw data signals to generate a plurality of derived moving averages thereof whereby 
to filter out large and small scale variations respectively; generating from the derived 
moving averages a composite signal including weighted moving averages as 
components thereof; and generating from the said composite signal an audible signal 
in which each amino acid is represented by an identifiable musical note. 

The present invention may be embodied in apparatus including visual display means by 
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which the raw data positional coordinate signals may be displayed graphically in 
synchronism with the audible output signals such that visual and aural correspondence 
is discernible. 



The present invention also comprehends apparatus for producing an audibly 
discernible representation of a three-dimensional structure, comprising: 

means for analysing numerical information to generate representations of the 

position of identifiable components thereof; 

means for forming moving averages of the said numerical information, 
- • means for converting the moving averages of the numerical information into 
scalar values, and 

means for allocating pitch or volume values to the said scalar values to 
generate audible sounds related to the structure. 

Various examples of the invention will now be more particularly described. 

Example 1: for the generation of musical notes representing the individual amino acids 
of a protein structure, each x, y and z positional coordinate of the CA carbon atom 
from each amino acid in an original protein sequence SO derived from x-ray 
crystallography analysis is processed twice, once to extract a first moving average SI, 
calculated by averaging the values of the sequence SO over a range of five amino acids, 
and then a second time to extract a second moving average S2 calculated by averaging 
the values of the moving average SI over a range of twenty five amino acids. The first 
moving average represents the secondary structure of the protein and the second 
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moving average represent the tertiary structure. 



From these three digital data streams S 50, 51, 52 may be derived three further 
sequences as follows: 

1 SO-1 is calculated by subtracting the first moving average SI from the original 
three dimensional data SO. This has the effect of emphasising the small twists and turns 
of the protein and of masking the larger scale curves over the protein structure as a 
whole. This in effect acts as a high frequency filter on the three-dimensional structure 
of the protein and highlights the spiral shape of the alpha helices. By subtracting the 
first moving average from the original data all the short-term fluctuations are 
maintained within the structure of the alpha helices without interference from larger 
scale changes. By mapping this to pitch variation an accurate sonic representation of 
the alpha helices is obtained. This mapping results in the generation of sequences of 
arpeggios when the alpha helices are traversed, and the same or a slightly varying note 
is heard when the beta sheets are traversed. This latter results from the fact that a beta 
sheet contains few small-scale changes. 

SI -2 is created by subtracting the second moving average from the first moving 
average. This has the effect of amplifying the medium sized turns of the protein whilst 
attenuating the higher frequency alpha helix turns and the lower frequency large scale 
curves. This process acts as a medium frequency filter on the three-dimensional 
structure. 

Finally, S2-3 is derived by calculating the mean position of all the amino acids in the 
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raw data SO to create a second moving average by subtracting this mean from each 
value of a coordinate for an amino acid in the moving average S2. The resulting 
sequence S2-3 is essentially identical to the sequence S2 but is normalised about a 
central zero origin point. This sequence represents the larger scale curves of the 
original three-dimensional protein structure, effectively acting as a low frequency filter. 

Weighting factors are then applied to each sequence and the weighted sequences 
summed to give a final sequence ST = a (SO-1) + b(Sl-2) + c (S2-3). The magnitude 
of the weighting factor a determines the proportion of the high frequency component 
of the protein in the final sequence ST whilst b and c determine the mid frequency and 
low frequency component. 

The sequence ST is than mapped onto musical parameters such as pitch via a three- 
dimensional waveform generated by three-dimensional spectral synthesis, thus creating 
sequences of notes which express visual features in musical form. The waveform maps 
a three-dimensional coordinate value onto a one-dimensional value, that is the value 
of that point in the waveform. The three dimensional input values are a sequence of 
coordinate values from ST and the output is a sequence of one-dimensional values 
which are interpreted as notes in musical scales or dynamic levels (volume). This 
mapping may be achieved as follows. First a random number (ranging from 5 to 20 or 
more) of vectors are provided which reside in a 2-dimensional frequency-space (the 
frequency-space is the spectral analog to the final 2 dimensional landscape). Each 
vector has a randomly determined position in this frequency-space and has a randomly 
determined phase and magnitude associated with it (represented by a complex number). 
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For mathematical reasons, there must also exist for each vector a 'mirror' vector with 
the same vector location but having a complex number associated with it that is a 
complex conjugate of the original vector's complex number. Therefore there must 
always be an even number of these vectors. 

Once these vectors with their associated complex numbers have been generated it is 
a simple process of fourier synthesis so that given a 2-dimensional coordinate in 
cartesian space, one can sum the contributions of each vector (at the cartesian 
coordinate) to end up with a scalar (1 -dimensional) value which can be interpreted as 
a height value. Details are given by D. Saupe. Since the resulting "landscape" is 
constructed from a combination of waves, for a small list of vectors it has the property 
that two points close to each other in 2-dimensions have similar resulting height values 
(i.e. mathematically speaking, the landscape is continuously differentiable everywhere. 

This process is extended to 3-dimensions by generating a list of 3-dimensional vectors 
in frequency -space. When given a point in 3-dimensional cartesian space, these 
vectors can be summed using complex arithmetic to provide a 1 -dimensional scalar 
value. This allows a 3-dimensional coordinate (i.e. 3-dimensional position of a CA 
atom after the moving average process) to be turned into a 1 -dimensional scalar value. 
In an analogous way to the 2-dimensional synthesis, two points close to each other in 
3-dimensional cartesian-space will have close 1 -dimensional results from the mapping. 
This means that close 3-dimensional positions will have close (audibly related) pitch, 
amplitude or other such musical value. 
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Whilst this invention may make use of a fourier synthesis technique, fourier synthesis 
by such is not new, and this invention includes the use of any other method of mapping 
a 3-dimensional point, for example derived from filters or moving averages of x-ray 
crystallography data to a 1 -dimensional point which can be interpreted in a musical 
fashion. A simple example might be simply to sum together the x, y, and z values of 
the 3-dimensional coordinate to get a single 1 -dimensional value. Another alternative 
in the specific case of proteins may be to use the coordinates of the second moving 
average as an "axis" and to calculate "angular" offsets of the original data from that 
axis. As the alpha helix twists around a sequence of angles averaging around 360° may 
then be derived. 

This output is then written to a MIDI text file for conversion into music or a layer in 
a musical arrangement using known MIDI music software to produce complex multi 
timbral musical pieces containing melodies reflecting the large-scale tertiary structures 
or small repetitive arpeggios expressing the structural nature of the alpha helices. 

Example 2 : for the production of a musical representation of a protein. This follows 
the same basic method as described in relation to example 1 above, with the addition 
of tempo, rhythm and key changes derived from the x-ray crystallography coordinate 
data. 

In this example positions in the original data sequence are determined where the 
musical parameter changes are to take place corresponding to visual transmission 
points in the secondary structure of the protein, for example where an alpha helix, a 
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beta sheet, a loop or a turn begins or ends. 

The secondary structure details are identified using a filter. This filter outputs 
secondary structure features from an input comprising the three-dimensional data. 
Two filter techniques may be used in the identification of the secondary structures, 
namely: 

Filter technique 1 . The three dimensional distance of each CA carbon atom is taken 
from the mean position of the CA carbon atoms of a particular section of the protein. 
The filter moves a window containing data covering approximately 10 CA carbon 
atoms for different amino acids along a section of the crystallography data looking at 
the distance of each CA carbon atom from the mean value of all the carbon atoms in 
the current window. From these distances it is possible to determine the likelihood of 
the current window containing data from an alpha helix or a beta sheet. The 
termination points of these features may thus be identified. 

Filter technique 2. In this technique matched filters are used with parameters tuned to 
those found in protein alpha helices and beta strands and the like. For example, alpha 
helices will have an approximate pitch of 4.6 angstrom units, beta sheets will have an 
approximate pitch of 13.5 angstrom units. 

Example 3. This example may be embodied in a model giving participants the 
opportunity to create their own pieces of music from a range of onscreen three 
dimensional proteins. The three dimensional x-ray crystallography x, y, z coordinate 
data is used as the basis for an animated display of the progress of the generated music 
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along the physical structure of the proteins as displayed graphically on a computer 
screen. By choosing from a selection of moving average operations the user can see 
how the chosen proteins are converted into musical sounds. The weightings a, b, c of 
example 1 may be made available to the user for selection, and their combination by 
the computer then results in a sequence of notes. 

Each note, corresponding to a specific amino acid in the display, is then sounded with 
the amino acid on the display, with the visual representation of the protein being 
highlighted simultaneously whereby to demonstrate and emphasise the correspondence 
between the musical notes and the protein sequences. 

Example 4: a display booth or exhibition kiosk may be equipped with an interactive 
touch screen compute^ system containing both the audio and visual components 
described in relation to examples 1 and 3. The screen may display graphics generated 
using a graphics processor depicting a traverse through the biological material of a 
plant, animal, insect, fungal, bacteria, yeast or human "landscape". The protein 
molecules are used to derive musical sounds identifying the proteins. This may also 
have an educational value in enabling aural recognition of proteins. 

Example 5. Another educational application may be developed as a three dimensional 
tactile model of a protein containing a plurality of switches connected to a computer 
system whereby the display of an audio visual information device may be influenced. 
On touching a particular amino acid (that is operating the associated switch) the amino 
acid on the display screen would be highlighted and the corresponding note, chord or 
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other musical sound representing this amino acid generated for the user to hear. 

The present invention thus expands and crosses the boundaries between science and 
art, enabling new approaches to understanding biological structure to be achieved as 
5 well as the generation of music for educational or purely entertainment purposes. Use 
of this invention in protein homology modelling, where fast and easily recognisable 
differences in proteins of similar structure are required, may be developed as there is 
a currently unfulfilled demand for such a tool in this widely used research technique. 

10 Protein-derived music may be of therapeutic interest in the treatment of conditions 
currently treated using music therapy. It is believed that such specialised music may 
offer a focused and precise application which optimises and hence is more cost 
effective than, known music therapies utilising music derived from other sources. 
Protein-derived music may be used to strengthen mind-body connections. Likewise, 

1 5 protein-derived music may be used for studying and modelling protein sequence data, 
complementing existing techniques which have wide application in drug design. Since 
alpha helices, beta strands and turns each have characteristic combinations of 
hydrophobic and hydrophilic amino acids, different structural categories of proteins, 
which combine these secondary elements in different ways, also have different musical 

2 0 characteristics. The sounds generated therefrom may therefore be used effectively in 
the discrimination and demonstration of different protein types. 

Protein music may also be used in sequence alignment and structural homology 
determination as structural homology may be heard when it might not easily be 
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perceived visually. 
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CLAIMS 

1 A method of producing an audibly discernible representation of a three 
dimensional structure, comprising the steps of generating numerical information 
concerning the spatial positions of identifiable components of the structure from 
analysis of the structure, converting the numerical information into a derived digital 
signal by forming moving averages of the numerical information, subjecting the 
derived digital signals to conversion from a three dimensional value to produce a 
signal having a scalar value representing at least one of the properties of an audibly 
discernible sound, and generating a sound having that or those properties. 

2. A method as claimed in claim 1, in which the structure is a protein and the 
analysis of the structure is performed by x-ray crystallography to determine the 
relative locations of the amino acids of which the protein is composed. 

3. A method as claimed in claim 2, in which the results of the x-ray 
crystallography analysis comprises a set of position coordinates representing the 
relative positions of at least some of the atoms of the amino acids of the protein. 

4. A method as claimed in claim 3, in which the position coordinate data 
represents the three dimensional position in space, with respect to a given frame of 
reference, of a carbon atom of the amino acids of the protein. 
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5. A method as claimed in any preceding claims, in which the said moving 
averages comprise at least a first moving average of the value of positional 
coordinate data scanned over successive groups of carbon atoms to provide an 
approximate representation of the secondary structure of the amino acids of the 
protein. 

6. A method as claimed in claim 5, in which a second moving average of values 
of the first moving average is taken over a range of values thereof greater than the 
range used to form the first moving average. 

7. A method as claimed in claim 6, in which there is formed a first further 
sequence of digital signals derived from said first and second moving averages by 
determining the difference between the first moving average and the positional 
coordinate data, whereby to provide a digital data signal in which larger scale 
variations are not present and smaller scale variations are amplified. 

8. A method as claimed in claim 6 and claim 7, in which a second further 
sequence of digital signals are derived from the first and second moving averages by 
determining the difference between the first and second moving averages whereby 
in effect to act as a mid-frequency filter on the data. 

9. A method as claimed in any of the claims 6 to 8, in which a third further 
sequence of digital signals is derived by normalising the second moving average 
values about a selected origin to represent the large scale features of the positional 
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coordinate data stream, acting effectively as a low frequency filter. 

10. A method as claimed in Claim 9, in which the said first, second and third 
further sequences of digital signals are each multiplied by a respective weighting 
factor (a, b, c) and summarised to produce a single sequence (ST) averaged and 
processed three-dimensional coordinate values which are then mapped onto a 
sequence of one-dimensional values via a three-dimensional waveform generated by 
spectral synthesis. 

11. A method as claimed in Claim 10, in which the one-dimensional value is 
pitch. 

12. A method as claimed in Claim 1 1, in which the pitch signal represents a 
musical chord or a single note. 

13. A method as claimed in Claim 10, in which the one-dimensional value is 
volume. 

14. A method as claimed in any preceding claim, in which the identification of 
the details of secondary structures in the positional coordinate data is obtained by 
means of a filter. 

15. A method of producing audibly recognisable sounds representing respective 
amino acids in a protein structure, comprising the steps of: subjecting the protein 
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structure to an x-ray crystallography analysis to produce a series of raw data signals 
representing the three-dimensional positional coordinates of the CA carbon atoms 
of the amino acids of which the protein is composed; filtering the said raw data 
signals to generate a plurality of derived moving averages thereof whereby to filter 
out large and small scale variations respectively; generating from the derived 
moving averages a composite signal including weighted moving averages as 
components thereof; and generating from the said composite signal an audible signal 
in which each amino acid is represented by an identifiable musical note. 

16. A method as claimed in Claim 15, in which the raw data positional 
coordinate signals are also displayed graphically on a display screen in 
sychronisation with the audible output signals such that visual and aural 
correspondence is discernible. 

1 7. Apparatus for producing an audibly discernible representation of a three- 
dimensional structure, comprising: 

means for analysing numerical information to generate representations of the 
position of identifiable components thereof; 

means for forming moving averages of the said numerical information, 
means for converting the moving averages of the numerical information into 
scalar values, and 

means for allocating pitch or volume values to the said scalar values to 
generate audible sounds related to the structure. 
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18. Apparatus as claimed in Claim 17, in which there are further provided means 
for displaying a two-dimensional visual representation of a three-dimensional 
structure, an interface with a user to allow selections of identifiable components of 
the structure to be made, and means for visually correlating the audible sounds 
produced by the apparatus with the displayed components. 

19. Apparatus as claimed in Claim 1 8, further comprising means for identifying 
a plurality of components of a structure in sequence and generating a sequence of 
sounds in correspondence therewith. 

20. Apparatus as claimed in Claim 1 7, further including a three-dimensional 
model of the structure, at least some of the said identifiable components thereof 
having associated switching means operable by a user in making selections of the 
sounds to be generated by the apparatus. 
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